1. Field of the Invention
The present invention relates to an image processing apparatus and an image processing program that are mounted in a digital camera or the like. In particular, the present invention relates to a technology for restoring an image that is blurred due to camera shake or the like.
2. Description of Related Art
In recent years, as digital cameras achieve a smaller size and a higher zooming factor, a camera shake correction has become necessary in many cases.
The system of camera shake correction includes an optical camera shake correction system and an electronic camera shake correction system. In the optical camera shake correction system, an optical system is controlled according to the amount of camera shake, thereby keeping an image forming position constant on an imaging device. On the other hand, in the electronic camera shake correction system, at the time of signal processing of an image formed on an imaging device in the state where the camera is shaken due to hand movement, a part of the image is cut out according to the amount of camera shake so that a subject always is displayed at the same position.
The electronic camera shake correction system is effective in the case of moving image capturing where images are captured successively, whereas the optical camera shake correction system has to be employed in the case of still image capturing.
However, since the optical system is controlled in the optical camera shake correction system, such a digital camera has a larger body than that not having means for the optical camera shake correction. Also, even when the optical camera shake correction is employed, it sometimes is impossible to follow all of the actual camera shake due to mechanical constraints. Accordingly, the camera shake is not always corrected completely, so that a camera shake component remains in some cases.
In response to this, an image processing apparatus is suggested for detecting a camera shake component remaining in an input image and performing signal processing to correct camera shake. For example, JP 11(1999)-284944A discloses the configuration in which an image is corrected by signal processing.
In the following, a conventional image processing apparatus (for example, that disclosed in JP 11(1999)-284944A) will be described with reference to FIGS. 19 and 20.
FIG. 19 is a block diagram showing a configuration of the conventional image processing apparatus. In FIG. 19, an image signal that is recorded by a camera or the like and converted to digital form is input to an input portion 101. The image signal to be input to the input portion 101 may be an image signal of an image captured by a digital camera or an image signal that is obtained by scanning a photograph taken using a film camera and converting the data to digital form. A CPU 102 controls an operation of the image processing apparatus. An image memory 103 temporarily stores the image signal input to the input portion 101, provides the image signal to the CPU 102 according to the control of the CPU 102, and stores again an image signal processed by the CPU 102.
Constituent components in FIG. 19 are similar to those in a general purpose computer, and the conventional image processing apparatus also can be embodied in the general purpose computer.
FIG. 20 is a flowchart for describing a camera shake correction method in the conventional image processing apparatus.
In the conventional image processing apparatus, first, the CPU 102 reads out an image signal (an image signal of the image that is blurred due to camera shake) stored in the image memory 103 and divides the image based on the read-out image signal into small block regions (S1). For example, the image based on the image signal read out from the image memory 103 is divided into small block regions of 3×3 or 5×5 dots.
Next, the CPU 102 performs edge detection for each of the small block regions of the image and determines the small region having the largest edge amount to be a “feature region” (S2). Herein, the “edge amount” is obtained by integration of an edge signal obtained by the edge detection over the small region. The edge amount increases with more minute edges in the image and decreases with fewer edges therein, which is a flat image.
Thereafter, the CPU 102 performs a two-dimensional filter processing of the image signal of the image in the feature region, thereby detecting a camera shake direction (S3).
Subsequently, the CPU 102 performs an image computation processing of the image signal of the image in the feature region while varying an assumed camera shake amount, thereby detecting a camera shake amount (S4).
Then, the CPU 102 determines a degradation function from the camera shake direction and the camera shake amount that have been detected (S5).
Next, the CPU 102 divides an image based on the image signal input to the input portion 101 into small block regions (S6).
Thereafter, the CPU 102 restores a pixel signal based on the degradation function obtained in S5 (S7).
Subsequently, the CPU 102 combines the image signals of the small regions whose pixel signal has been restored, thus restoring an overall image signal (S8). The image signal restored in the CPU 102 is output from an output portion 104.
However, although the processing of extracting the feature region (S2) disclosed in JP 11(1999)-284944A extracts the small region having a large edge amount as the feature region by performing the edge detection, the frequency characteristics of an image do not necessarily depend on the camera shake direction. For example, the result of detecting camera shake direction in the case where the small region whose vertical line has a large edge amount is extracted as the feature region sometimes is different from that in the case where the small region whose horizontal line has a large edge amount is extracted as the feature region. In the former case, the camera shake amount can be detected for horizontal camera shake but cannot be detected for vertical camera shake due to its high image correlation.
As described above, the image processing apparatus disclosed in JP 11(1999)-284944A sometimes cannot detect the camera shake amount, leading to a problem that it is not possible to restore the image signal input to the input portion 101 properly.